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Overview 



In 2004, Lumina Foundation for Education launched “Achieving the Dream: Community Colleges 
Count,” a national initiative aimed at improving success among community college students, particularly 
low-income students and students of color. Now encompassing more than 130 institutions in 24 states and 
the District of Columbia, Achieving the Dream helps community colleges build a “culture of evidence” by 
using student records and other data to examine students’ performance over time and to identify barriers to 
academic progress. From there, community colleges are expected to develop intervention strategies 
designed to improve student outcomes; conduct further research on student progress; and bring effective 
programs to scale. As a result, it is anticipated that colleges will see measurable improvements over time 
in student outcomes, including increased progress through developmental education and college-level 
“gatekeeper” (introductory) courses, grades, persistence, and completion of credentials. 

This report examines the first 26 colleges to join Achieving the Dream in 2004 (known as the “Round 
1” colleges), and tracks their progress through spring 2009. The key findings are: 

• Four out of five Round 1 colleges adopted practices associated with a moderate to strong 
culture of evidence. These colleges made important enhancements to their institutions, includ- 
ing more sophisticated methods for data analysis and more efficient systems for monitoring 
their efforts to improve students’ achievement. Conversely, about one-fifth of the colleges still 
struggled to implement many of the initiative’s recommended practices, hindered primarily by 
weak institutional research capacity. 

• Colleges that made the greatest strides shared several key characteristics, including broad- 
based involvement of college administrators, faculty, and staff; strong institutional research de- 
partments that produced accessible reports on student achievement; regular evaluations of their 
programs; and scale-up of successful programs. 

• Colleges instituted a wide range of strategies to improve student achievement, but a ma- 
jority of them remained small in scale. The most popular strategies were tutoring, supple- 
mental instruction, advising, success courses, and learning communities. However, a majority 
of these reforms reached less than 10 percent of their intended target populations. 

• Achieving the Dream had an important i nf luence on most colleges. Representatives from 
three-fourths of the colleges said that the initiative had at least some influence on their devel- 
opment of a culture of evidence. Other important influences included accreditation systems, 
grants in addition to those from Achieving the Dream, and visionary college leaders. 

• Trends in student outcomes remained relatively unchanged, with a few exceptions. On aver- 
age, after Achieving the Dream was introduced, colleges saw modest improvements in the per- 
centage of students completing gatekeeper college English courses and courses completed. In 
contrast, students’ persistence and the percentage of students completing developmental math, 
developmental English, developmental reading, and gatekeeper math courses remained substan- 
tially the same. 



A final report on the Round 1 colleges, with a follow-up on the trends in student outcomes, is planned 
for 2012-2013. 




Preface 



Today, more students than ever are enrolling in community college, but nearly half of them never go on 
to earn a degree or certificate. Until recently, most community colleges did not have the capacity or the 
incentives to gather information about their students’ performance over time and the possible barriers to 
their success. Traditionally focused on increasing access to postsecondary education, particularly for 
low-income students, community colleges are now turning more attention to improving the academic 
success of their students. But without concrete evidence to illuminate how and why their students are 
underperforming, colleges have been hard-pressed to take steps to help them improve. 

Enter “Achieving the Dream: Community Colleges Count,” an ambitious national initiative launched by 
Lumina Foundation for Education in 2004, with support from a number of other philanthropies and 
organizations. The initiative helps community colleges learn how to collect and analyze student 
performance data in order to build a “culture of evidence” — a culture in which colleges routinely use 
solid evidence to develop institutionwide reform strategics that arc aimed at helping their students 
succeed academically. Achieving the Dream has grown into a national movement; more than 130 
community colleges are now participating. 

This interim report takes stock of the experiences among the first 26 colleges from five states to join the 
initiative (the “Round 1” colleges). It offers a look at the influence of the initiative in its early stages, while 
its practices and supports were still evolving. The study shows that after five years in Achieving the 
Dream, four out of five of the 26 Round 1 colleges had adopted some or all of the practices associated 
with a culture of evidence. While the colleges instituted a wide range of strategics to help their students 
succeed, increasing student achievement and persistence levels in the colleges remains a huge chal- 
lenge. 

What are we to make of Achieving the Dream at this juncture? On the one hand, it represents an unprece- 
dented effort to help community colleges change their institutional cultures so they can continuously 
pinpoint where, how, and why their students are underperforming, and then implement reforms to help 
them improve. Achieving the Dream was at the vanguard, anticipating — and helping spur — the growing 
national focus on institutional accountability in community colleges. After five years in the initiative — 
the first two years of which were spent planning and setting up strategies for reform — a majority of the 
Round 1 colleges are collecting data on their students’ performance and are committed to using those data 
to design and implement effective reforms. On the other hand, t lends in student outcomes — such as 
course completion, persistence, maintaining good grades, and earning college credentials — have re- 
mained relatively unchanged at these colleges, underscoring the depth of the challenge facing all commu- 
nity colleges. 

One lesson from Achieving the Dream is that simply understanding the problems is not enough to solve 
them. Nevertheless, Achieving the Dream has provided a better sense of where the colleges have to go to 
improve and how to get there, creating an infrastructure through which change can occur. What remains is 
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for colleges to harness the potential power of that infrastructure — and for Achieving the Dream itself to 
further refine its approach in order to better help colleges focus systematically on the issues that affect 
student achievement. 

Achieving the Dream is not a static intervention, but a living, organic process — and many of the Round 1 
colleges are now moving into the next stage of that process. A final report on the Round 1 colleges is 
planned for 2012-2013, by which time they will have had more time to institutionalize their reforms. 

Gordon Berlin 
President 
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Executive Summary 



Across the United States, enrollments at community colleges are soaring. In fall 2009, 
approximately 8 million students took courses for credit — nearly a 17 percent increase from 
two years before. 1 Community colleges tend to be welcoming and accessible institutions, par- 
ticularly for nontraditional, low-income, and minority students. But while community colleges 
do a good job of helping students get a foothold in higher education, close to half of the students 
who begin at community colleges with the intention of earning a certificate or degree do not 
achieve their goal and are not enrolled in any college or university after six years. 2 A growing 
number of policymakers and educators are calling upon community colleges to do better. 

“Achieving the Dream: Community Colleges Count” is a national initiative that was 
launched in 2004 and was designed to increase the academic success of community college stu- 
dents, with a special emphasis on low-income students and students of color. Its approach is to 
help community colleges build a “culture of evidence” by using student records and other data 
to examine how students are performing over time and to identify barriers to academic progress. 
From there, community colleges are expected to develop intervention strategies designed to im- 
prove student outcomes, conduct further research on student progress, and bring effective pro- 
grams to scale. Lumina Foundation for Education launched Achieving the Dream and was its 
principal funder, with a large group of philanthropies and other organizations joining Lumina to 
support the initiative. Today, more than 130 community colleges in 24 states and the District of 
Columbia are involved in Achieving the Dream (see Figure ES.l). 

MDRC — a nonprofit, nonpartisan education and social policy research organization 
— and the Community College Research Center (CCRC) are evaluating the work of the first 26 
colleges to join the initiative from Florida, New Mexico, North Carolina, Texas, and Virginia 
(called the “Round 1” colleges here). This report covers a five-year period, beginning in the 
2004-2005 academic year. Because the process of building a culture of evidence is envisioned 
as a long-term effort, a future report will examine trends in student achievement after the col- 
leges have had more time to institutionalize reforms that began under Achieving the Dream. 

Overall, the report concludes that: 



'American Association of Community Colleges (AACC), Community College Enrollment Survey: An 
Analysis of Estimated Fall 2009 Headcount Enrollments at Community Colleges (Washington, DC: AACC, 
2009). 

2 Stephen Provasnik and Michael Planty, Community Colleges: Special Supplement to the Condition of 
Education 2008, NCES 2008-033 (Washington, DC: National Center for Education Statistics, Institute of Edu- 
cation Sciences, U.S. Department of Education, 2008). 
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Achieving the Dream: Community Colleges Count 



Figure ES.l 

States and Colleges Involved in Achieving the Dream, 2004 to 2010 




■ Round 1 Achieving the Dream states (joined in 
2004-2005) 

H Achieving the Dream states that joined after 2005 

A Round 1 Achieving the Dream colleges (joined in 
2004-2005) 

• Colleges that joined Achieving the Dream after 2005 




NOTE: Multi-college districts are represented with one dot on this map. 
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• Many Round 1 colleges made important strides in building a stronger culture of evi- 
dence during their five-year participation in the initiative. In particular, colleges en- 
hanced their leadership commitment to student success, increased their research capaci- 
ty, and developed a number of interventions aimed at improving their students’ 
achievement. 

• By spring 2009, four out of five Round 1 colleges had adopted a number of practices 
associated with a strong culture of evidence, with one-fifth of the colleges still facing 
major challenges in undertaking this work. 

• While colleges instituted a wide range of strategies to improve student achievement un- 
der the auspices of Achieving the Dream, a majority of these reforms reached less than 
10 percent of their intended target populations — likely too few to make demonstrable 
progress on improving student achievement overall. 

• Other forces besides Achieving the Dream sometimes helped the colleges build a cul- 
ture of evidence, including accreditation processes and grants other than those from the 
initiative. However, about three-fourths of the colleges indicated that Achieving the 
Dream had at least some influence in helping them develop a culture of evidence. 

• Trends in student outcomes remained relatively unchanged, except for modest im- 
provements in gatekeeper (introductory) college English courses and the completion of 
courses attempted within the first two years. 



The Theory of Action Behind Achieving the Dream 

From the earliest conversations, it was clear that Lumina Foundation and the partners 
had big ambitions for Achieving the Dream. They spoke about fostering fundamental changes 
in the culture and operations of community colleges — changes that would lead to measureable 
and lasting improvements in student outcomes. They also spoke about changing the context in 
which community colleges operate — everything from the rules governing their accreditation 
and funding to public awareness and support for community colleges. 

On the ground, participating colleges were expected to enact a five-step process of insti- 
tutional reform, which included (1) securing leadership commitment; (2) using data to prioritize 
actions; (3) engaging stakeholders; (4) implementing, evaluating, and improving intervention 
strategies; and (5) establishing a culture of continuous improvement. To help colleges institute 
their reforms, Achieving the Dream provided them with both monetary and technical support. 
The Round 1 colleges were awarded an initial year-long planning grant of $50,000 each, fol- 
lowed by annual grants of $100,000 for four years ($450,000 total). In addition, the colleges 
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were aided by two consultants: a data facilitator, who helped them perform the data collection 
and analysis and interpret the results; and a coach, who helped them set priorities, build consen- 
sus, and implement strategies for improvement. The initiative also sponsored a Kick-off Insti- 
tute and annual Strategy Institutes for all the Achieving the Dream colleges, where they learned 
more about the Achieving the Dream process, made plans for their own campuses, and shared 
ideas and lessons with other colleges on how to help students be more successful. Colleges also 
received technical assistance from the initiative’s eight founding partner organizations, led by 
MDC, Inc., the managing partner of Achieving the Dream. 

All Achieving the Dream colleges were asked to try to “move the needle” on five mea- 
sures of student success: (1) completion of developmental courses and progression to credit- 
bearing courses; (2) completion of so-called gatekeeper courses, including introductory college 
courses in English and math; (3) completion of attempted courses with a grade of “C” or better; 
(4) persistence from semester to semester and from year to year; and (5) attainment of college 
credentials. Colleges were expected to track these outcomes for each fall cohort of entering stu- 
dents and to make comparisons with past cohorts to determine whether the outcomes improved 
over time. As noted above, colleges were also expected to break down the results for subgroups 
of students defined by race, ethnicity, and income. 

Throughout the history of Achieving the Dream, there was some tension between those 
who believed that gains in student outcomes should come quickly and those who believed it 
was a long-term process. The initiative’s “Integrated Action Plan” was vague on this point, in- 
dicating that colleges should see “measurable improvement in success rates” after four years 
and have “achieved their long-term targets for student success” after eight years. What was clear 
was that the five-step process was iterative, so any improvements in student outcomes depended 
on the colleges’ ability to master the initial steps. Even if colleges did all that was asked of 
them, it could still take time for improvements to show up in the data, given the longitudinal 
nature of the measures. Some of the partners cited the oft-used metaphor of turning a ship to 
describe the work of Achieving the Dream. On the one hand, they firmly believed that the initia- 
tive would set community colleges on a better course; on the other hand, they recognized that 
changes would likely occur only gradually and might not be fully apparent for several years’ 
time. 



In analyzing the work and progress of the Round 1 colleges, it is also important to note 
that these schools began their work in Achieving the Dream as the initiative was still evolving. 
Indeed, since its beginning in 2004, the initiative has expanded, modified, and codified many of 
its practices and supports, with the Round 1 colleges receiving less intensive versions of these 
messages during the early phases of their implementation. As such, the Round 1 colleges’ 
progress should be seen as offering a look at the initiative’ s influence during its early develop- 
ment, while its practices and supports were still being solidified. 
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The Round 1 colleges are diverse in size, location, and student characteristics. The larg- 
est institution is Houston Community College, which had a full-time equivalent 12-month 
enrollment of over 32,000 students in 2008-2009. The smallest institution is Martin Community 
College in North Carolina, which had a full-time equivalent enrollment of 410 students. The 
colleges are located in large and midsize cities, suburbs, and small towns. White students make 
up a majority or plurality of students at most of these institutions, but nearly all the colleges 
enrolled substantial numbers of African American, Hispanic, or Native American students. 



How Was the Research Conducted at the Round 1 Colleges? 

A research team comprising MDRC and CCRC staff studied the Round 1 colleges over 
several years to learn how they implemented Achieving the Dream and whether they made 
progress in developing a culture of evidence on campus. The team visited each of the 26 colleg- 
es two times: once during the first year of implementation, in spring 2006, and again during the 
last year of implementation, either in fall 2008 or spring 2009. The visits lasted two to three 
days and included interviews with the administrators, faculty members, and institutional re- 
search staff, among others. The research team also conducted a survey of college administrators 
and faculty at 23 of the 26 colleges in 2007-2008. In addition, the research team reviewed an- 
nual reports that the colleges submitted to Achieving the Dream, as well as reports prepared by 
the coaches and data facilitators. Finally, the research team analyzed student records data that 
the colleges submitted to the Achieving the Dream database, which allows researchers to track 
student performance over time. The analysis examines student outcomes at the institutional lev- 
el for students who began college each fall between 2002 and 2007 (three years before the initi- 
ative began and three years after its implementation). Each entering cohort is tracked for two 
years. 



Have the Round 1 Colleges Created a Culture of Evidence? 

The bulk of this report assesses the Round 1 colleges’ success between spring 2006 and 
spring 2009 in developing a culture of evidence — that is, their progress in instituting the five 
main steps of the Achieving the Dream model, described above. 

As Table ES.l shows, most of the Round 1 colleges made substantial progress in devel- 
oping more evidence-based systems aimed at improving student success during their participa- 
tion in Achieving the Dream. Eighty-one percent (21) of the Round 1 colleges improved their 
culture of evidence over the course of their five-year participation in the initiative, and 1 9 per- 
cent (five) of the colleges continued to have major challenges implementing the initiative’s 
model for institutional improvement. Indeed, most of the Round 1 colleges made a number of 
enhancements to their institutions, including the use of more sophisticated methods for data 
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Achieving the Dream: Community Colleges Count 
Table ES.l 



Culture of Evidence at Achieving the Dream Round 1 Colleges, 
Spring 2006 and Spring 2009 





Spring 2006 


Spring 2009 


Culture of Evidence Rating 


Number of 
Round 1 Colleges 


Percentage of 
Round 1 Colleges 


Number of 
Round 1 Colleges 


Percentage of 
Round 1 Colleges 


Strong or very strong 
culture of evidence 


6 


23 


11 


42 


Some culture of evidence 


5 


19 


10 


38 


Weak or very weak 
culture of evidence 


15 


58 


5 


19 


Total 


26 


100 


26 


100 



SOURCE: Information collected from faculty, staff, and administrators during two rounds of implementation 
research across all 26 Round 1 colleges, cross-referenced against reports submitted by college representatives, 
Achieving the Dream coaches, and data facilitators. 

NOTE: Percentages may not add to 100 because of rounding. 

analysis, numerous interventions aimed at increasing students’ success, and more efficient sys- 
tems for monitoring their efforts to improve students’ achievement. The colleges can be divided 
into three categories, based on their success in institutionalizing a culture of evidence. 

Strong Culture of Evidence 

Eleven Round 1 colleges (42 percent) had implemented most of the practices that are 
associated with a strong culture of evidence by the end of their five-year participation in 
Achieving the Dream. These colleges had excelled at building or further strengthening all five 
aspects of the initiative’s institutional improvement model — that is, they (1) had strong leader- 
ship systems committed to improving student success; (2) had well-developed institutional re- 
search departments that conducted intensive, in-depth analyses of their students’ achievement; 
(3) involved faculty, staff, students, and external stakeholders in multiple aspects of their reform 
agenda; (4) had used well-developed strategies, with one or more strategies reaching substantial 
proportions of their target population; and (5) had instituted multiple processes for strategic 
planning and program review, and made financial and programmatic decisions based on evi- 
dence of improved student achievement. As described in Box ES.l, the colleges that made the 
greatest strides in developing an evidence-based approach to improving student achievement 
shared several important characteristics, including the broad-based involvement of college ad- 
ministrators, faculty, and staff; strong institutional research departments that produced reports 
on student achievement that people could readily understand; regular evaluations of interven- 



6 








Box ES.1 



“Star” Colleges in Achieving the Dream: Institutions that Made 
Solid Progress in Building a Culture of Evidence 



Four of the 26 Round 1 colleges stood out because they had a weak culture of evidence at 

the start of the initiative and made great strides in building up their culture over five years. 

These colleges made changes that set them apart from the other colleges that made less 

progress, including: 

• Multiple senior administrators who were deeply involved in the colleges’ reforms. 

Each college had a cadre of senior leaders, including the vice presidents of instruction 
and student services and the head of institutional research (IR), leading its Achieving 
the Dream work. Two colleges also had presidents who were very involved. 

• Active IR departments that produced accessible reports on students’ achieve- 
ment. Each school focused sharply on increasing IR capacity by hiring new IR staff 
and upgrading its data management system, which allowed it to better manage multiple 
IR demands. Each school also produced institutional- or department-specific reports on 
students’ success, with clearly delineated target goals for students’ achievement. 

• Regular evaluations of their interventions to improve student success. Each col- 
lege systematically tracked its interventions, often comparing students who were par- 
ticipating in a particular program with nonparticipants. These findings were dissemi- 
nated widely throughout the school, with a number of faculty and staff commenting on 
the results. 

• Strong faculty and staff leaders who played an active role in leading the colleges’ 
reforms and interventions. Each of the four colleges had strong participation from 
faculty and staff, who were involved in leadership and policymaking committees at 
their colleges and led the implementation of numerous strategies that were aimed at 
improving students’ success. 

• Integrated committee structures that allowed for regular communication among 
administrators, faculty, and staff. Each of the four schools developed numerous 
standing committees to monitor its reform efforts. These committees brought faculty, 
staff, and administrators together through regular communication, providing a clear 
and consistent voice in their colleges’ larger policy- and institutional decision-making. 

• Consistent attention to financial supports for strategy scale-up and numerous 
strategies operating at scale. Each of these college developed systematic ways to 
support broad-scale reforms, often by using federal or state grants to support the addi- 
tional costs. All four colleges had strategies that were reaching substantial proportions 
of their remedial students or students who were in college for the first time. 

• Heavy investment in professional development. Each of the four schools dedicated 
significant funds to training faculty and staff. 
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tions to improve student success; and attention to scaling up program strategies that helped stu- 
dents be more successful. 

Some Culture of Evidence 

Ten Round 1 colleges (38 percent) had instituted many of the aspects of Achieving the 
Dream' s suggested improvement process in their schools, though not to the same degree as the 
colleges described above. Most schools had built (or started with) relatively capable institutional 
research departments, and most undertook longitudinal analyses of student outcomes and 
tracked the progress of at least some of their student success interventions. However, they also 
continued to have limitations in their data capacity that hindered their ability to undertake 
broader-based analyses. Other stakeholders, such as faculty and staff, also reported increased 
awareness of student achievement or had some exposure to reports on students’ success in par- 
ticular interventions. Faculty and staff were also generally involved in some aspects of the 
reform agenda at many of these schools, most often around implementing the colleges’ student 
success strategies, although their participation tended to be more limited than at colleges that 
had a better-developed culture of evidence. Finally, most colleges had some form of strategic 
planning process in place, which at times used student achievement data to inform decision- 
making, though the use of data was not consistent across these colleges. 

Weak Culture of Evidence 

As observed earlier, five colleges (19 percent) were still struggling to implement a 
number of Achieving the Dream’s recommended practices by spring 2009. Very weak institu- 
tional research departments were the primary difficulty hindering most of these schools’ ability 
to institute a broad data-driven culture. Most of these schools did not employ an institutional 
researcher continuously during their time in the initiative, relying instead on outside consultants 
or other, less well-trained staff to fill these positions. Sometimes these challenges were the re- 
sult of a poor economic situation, which kept the college from being able to attract good tech- 
nical support. Some schools also had difficulty implementing Achieving the Dream across mul- 
tiple campuses and providing sustained leadership to bring these disparate groups together. 



How Much Did Achieving the Dream Influence the Round 1 
Colleges? 

Many of the Round 1 colleges attributed their improvements to Achieving the Dream. 
As can be seen in Table ES.2, 19 out of 26 (73 percent) of the Round 1 colleges reported that 
Achieving the Dream had at least some influence on their development of a culture of evidence. 
However, they also noted other systems that were important in helping them develop a culture 
of evidence. For instance, a number of the schools discussed how their accreditation processes 




Achieving the Dream: Community Colleges Count 
Table ES.2 



Influence of Achieving the Dream on the Culture of Evidence at Round 1 
Colleges and Progress Made from Spring 2006 to Spring 2009 



Number of Colleges 


Spring 2006 
Culture of 
Evidence 3 


Spring 2009 
Culture of 
Evidence 


Progress Made 
from Spring 2006 
to Spring 2009 b 


Heavv influence: 8 colleges 


3 colleges 


Strong 


V ery strong 


+1 


1 college 


Some 


Strong 


+1 


3 colleges 


Weak 


Strong 


+2 


1 college 


Very weak 


Strong 


+3 


Some influence: 11 colleges 


2 colleges 


Strong 


V ery strong 


+1 


1 college 


Very weak 


Some 


+2 


6 colleges 


Weak 


Some 


+1 


2 colleges 


Very weak 


Weak 


+1 


Little influence: 7 colleges 


1 college 


Strong 


V ery strong 


+1 


1 college 


Weak 


Some 


+1 


2 colleges 


Some 


Some 


0 


1 college 


Very weak 


Very weak 


0 


2 colleges 


Some 


Weak 


-1 



SOURCE: Information collected from faculty, staff, and administrators during two rounds of 
implementation research across all 26 Round 1 colleges, cross-referenced against reports submitted 
by college representatives. Achieving the Dream coaches, and data facilitators. 

NOTES: While some of the colleges in the "Little influence" category made progress in developing 
their cultures of evidence, they did not consider Achieving the Dream to be a primary motivator for 
these changes. 

“Spring 2006 culture of evidence ratings are based on implementation findings from MDRC and 
CCRC's baseline report on the Round 1 Achieving the Dream colleges (Brock et al., 2007, pp. 96- 
97). 

b Progress ratings reflect changes in colleges’ cultures of evidence between spring 2006 and spring 
2009, as measured by movement across the following rating scale: very weak, weak, some, strong, 
very strong. For example, a college that received a "very weak" rating in spring 2006 and a "very 
strong" rating in spring 2009 would receive a progress rating of "+4," indicating that it had 
progressed 4 levels in the rating scale. 
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or grants other than those from Achieving the Dream, such as federal Title V or TRIO 
grants, were the main drivers of the reforms at their colleges. Achieving the Dream was 
seen as “complementing” or “reinforcing” this agenda, but not necessarily as the primary 
inspiration for the colleges’ efforts to improve student achievement or to use data. Addi- 
tionally, a few colleges said that the motivation for their reforms stemmed from an internal 
push to increase student achievement, often from a strong leader who was already commit- 
ted to improving student success. 



What Program Strategies Were Developed for Students, and to 
What Effect? 

Colleges can expect real changes in student outcomes only when they extend meaning- 
fully improved programs and services to significant numbers of students. In other words, a 
strategy will make an observable impact on institutional performance only if it (1) improves 
academic outcomes among the students who are served, and (2) reaches enough students to 
“move the needle” on collegewide measures. 

Overall, the Round 1 colleges implemented a large number of strategies under the aus- 
pices of Achieving the Dream. Most of those strategies involved the implementation of direct 
programs and services that were designed to improve students’ college success. While the strat- 
egies ranged widely, from light-touch orientation activities to more intensive curricular reforms, 
several common themes emerged across the 26 colleges: 

• A large majority of strategies were designed to increase academic and social 
support systems for students, while only about one-fourth changed the con- 
tent and delivery of classroom instruction itself. 

• Nearly half of the colleges’ strategies targeted developmental education stu- 
dents, and one -third focused on students in their first year of college. Very 
few strategies targeted students based on their race, ethnicity, or economic 
status, though some colleges addressed achievement gaps indirectly through 
programs aimed at a broader group of students. 

• Despite colleges’ notable efforts to scale up their programs and services, the 
majority of strategies reached less than 10 percent of their intended target 
populations. Intensive strategies, such as curricular reforms or intensive ad- 
vising, which were defined by longer periods of contact time with students, 
were particularly unlikely to reach large numbers of students. Student suc- 
cess courses (courses aimed at introducing students to college life and en- 
hancing their study skills) were the sole high-intensity strategy to reach a 
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large number of students at a majority of colleges. This finding suggests that 
colleges often faced a trade-off between the intensity and scale of their inter- 
ventions. 

Most colleges had several additional strategies that did not reach students directly, but 
rather through (1) changes in college policies, and (2) collaborative relationships with external 
stakeholders, including local school districts and community members. 

Besides improving upon the programs, services, and policies that affect students, 
Achieving the Dream colleges are expected to leverage engagement among faculty and staff 
and improve their skills as part of the effort to increase student success. Indeed, the initiative 
highlights the importance of professional development in creating more systemic changes in 
community colleges’ improvement efforts. This proved to be an important area of college ac- 
tivity under Achieving the Dream, with every college reporting at least one professional de- 
velopment strategy and several reporting that this was a key focus of their work. Findings 
from an MDRC cost study of Achieving the Dream further highlight the centrality of this 
work: investments in professional development averaged $654,000 across the five Round 1 
colleges in the study, representing an important share of colleges’ spending on activities re- 
lated to the initiative. 3 

On average, each college implemented seven strategies, with all 26 colleges implement- 
ing over 200 strategies in total, representing an enormous investment of time and work by col- 
lege personnel. Also, even though, as already noted, most strategies reached less than 10 percent 
of their intended target, the colleges made noteworthy progress toward scale-up, as nearly all 
succeeded at expanding at least one direct strategy to reach at least 25 percent of its intended 
target population. However, a majority of strategies across the Round 1 colleges remained small 
in scale, particularly when they involved the kind of intensive contact that might be expected to 
meaningfully influence students’ performance. The result is that the benefits of promising inter- 
ventions were frequently extended to only a fraction of the students who were in need of more 
intensive assistance. 

It should not be surprising then, that the trends in student outcomes across the 
Round 1 colleges remained relatively unchanged across the pre-initiative and post-initiative 
periods. The analysis suggests that the average rates of persistence and graduation as well 
as the rates for completion of developmental math, developmental English, developmental 
reading, and gatekeeper math courses remained substantially the same throughout the pe- 



3 Elizabeth Zachry and Erin Coghlan with Rashida Welbeck, Investing in Change: How Much Do Achiev- 
ing the Dream Colleges Spend — and from What Resources — to Become Data-Driven Institutions? (New 
York: MDRC, 2010). 
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riod, with modest improvements over time in the rates for completion of gatekeeper English 
and completion of coursework. 

In addition, few changes were seen in the outcomes of specific college and student 
subgroups. For instance, a separate analysis of the outcomes for several colleges that made 
the most progress in building a culture of evidence during their time in Achieving the 
Dream revealed no significant differences from the trends of the Round 1 colleges overall. 
Similarly, few changes were seen in the achievement gaps by race, ethnicity, and income. 
Though African American students showed some improvements in course completions and 
white students showed improvements in both course completions and gatekeeper English 
pass rates, those improvements over time were not large enough to close the achievement 
gap between the two groups. 

While these descriptive trends seem to suggest that Achieving the Dream had few posi- 
tive effects on student outcomes, the results must be interpreted with caution. While the lack of 
movement in student outcomes may occur largely because the interventions affected a small 
proportion of students in a cohort, the stability of student indicators over time may also reflect 
the short time span over which outcomes were analyzed. That is, the analysis of two-year out- 
comes for students may not provide a long enough window to adequately reflect the changes in 
institutional capacity and the implementation of specific strategies given the very low levels of 
student achievement that most colleges faced initially. 



Conclusion 

While community colleges are a prominent part of the national conversation today, few 
organizations were concentrating on these institutions before 2004, and even fewer were devot- 
ing substantial funds toward their improvement. Moreover, before Lumina Foundation for Edu- 
cation launched Achieving the Dream, few colleges or other postsecondary organizations fo- 
cused on community college students’ success; rather, they focused more often on providing 
open access to postsecondary education. 

In 2010, Achieving the Dream became an independent nonprofit entity, and it is under- 
going a marked shift in its leadership and organization as a result, seeking to involve an even 
wider range of community colleges in its work. In a change from the past, community colleges 
will have to pay to participate in Achieving the Dream, though it is expected that some institu- 
tions may find sponsors. Many of the core activities that Achieving the Dream provides — such 
as coaching, data facilitation, and annual Strategy Institutes — will also be subsidized so that 
colleges do not have to cover the full cost of participation. 
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As Achieving the Dream moves into this new phase, the initiative, like its participating 
institutions, may need to refine its approach to make a stronger impact on students’ success. For 
instance, outside of implementing colleges’ strategies, the faculty and staff at many of the 
Round 1 colleges tended to be less involved than administrators in the colleges’ overall im- 
provement work. Similarly, only a handful of colleges made attempts to engage their part-time 
faculty and staff, who most often lead the developmental and gatekeeper courses that are so im- 
portant to students' ultimate success in college. Additionally, while most colleges had expanded 
at least one strategy, the majority of strategies at these schools remained small in scale, leaving 
large proportions of students relatively untouched by the colleges’ Achieving the Dream work. 

These issues represent important challenges to Achieving the Dream's vision for im- 
proved student achievement; however, they should also be placed in the context of the overall 
struggles facing the U.S. education system as a whole. Elementary and secondary schools have 
long faced an uphill battle in their efforts to improve students’ achievement, with only a relative 
few achieving large-scale reforms that successfully increased students’ academic skills or grad- 
uation rates. Far more have been mired in many of the same struggles that Achieving the Dream 
colleges face, including students with major academic challenges, poor funding streams, and a 
lack of faculty or staff engagement in larger school reform efforts. Indeed, school systems 
working with adolescents, such as middle and high schools, tend to have the most struggles with 
large-scale reform, mirroring many of the key challenges that community colleges are facing. 

As Achieving the Dream continues to expand, it might look to revise its framework for 
action by drawing on some of the lessons from efforts by K-12 stakeholders to improve stu- 
dents’ outcomes. Some of these steps include a more rigorous focus on changing the practices 
that are most likely to affect students — namely, the interactions that occur between students 
and instructors in the classroom. As K-12 schools have learned, changing such practices can be 
difficult and often requires heavy engagement with faculty and staff, strong and continuous pro- 
fessional development, and a carefully researched plan detailing how new practices will affect 
students’ achievement. Given this scenario, Achieving the Dream might look toward more de- 
finitive ways to involve larger proportions of faculty and staff in the change process while also 
aiming to develop a larger evidence base about what practices work to increase community col- 
lege students’ achievement. Similarly, the initiative might also seek to incorporate classroom- 
based measures of learning, which more clearly document students’ attainment of particular 
skills and practices, in its model for institutional improvement. Such measures might help 
bridge the gap in Achieving the Dream's theory of action, which currently focuses on broad 
institutional changes in student outcomes that may take many years to manifest. 

While larger changes in students’ achievement may not yet have been realized, Achiev- 
ing the Dream has begun an unprecedented movement toward helping colleges improve their 
student outcomes and develop systems to sustain those efforts. Bringing faculty and staff voices 
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more concretely into colleges’ reform work and focusing more directly on improvements to 
classroom instruction and services may reap benefits for the next stage of the initiative’s work. 
Given the successes they have already had, the initiative and its participating colleges stand 
poised to move forward with this agenda and make the changes needed to help more communi- 
ty college students accomplish their goals. 
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About MDRC 



MDRC is a nonprofit, nonpartisan social and education policy research organization dedicated 
to learning what works to improve the well-being of low-income people. Through its research 
and the active communication of its findings, MDRC seeks to enhance the effectiveness of so- 
cial and education policies and programs. 

Founded in 1974 and located in New York City and Oakland, California, MDRC is best known 
for mounting rigorous, large-scale, real-world tests of new and existing policies and programs. 
Its projects are a mix of demonstrations (field tests of promising new program approaches) and 
evaluations of ongoing government and community initiatives. MDRC’s staff bring an unusual 
combination of research and organizational experience to their work, providing expertise on the 
latest in qualitative and quantitative methods and on program design, development, implementa- 
tion, and management. MDRC seeks to learn not just whether a program is effective but also 
how and why the program’s effects occur. In addition, it tries to place each project’s findings in 
the broader context of related research — in order to build knowledge about what works across 
the social and education policy fields. MDRC’s findings, lessons, and best practices are proac- 
tively shared with a broad audience in the policy and practitioner community as well as with the 
general public and the media. 

Over the years, MDRC has brought its unique approach to an ever-growing range of policy 
areas and target populations. Once known primarily for evaluations of state welfare-to-work 
programs, today MDRC is also studying public school reforms, employment programs for ex- 
offenders and people with disabilities, and programs to help low-income students succeed in 
college. MDRC’s projects are organized into five areas: 

• Promoting Family Well-Being and Children’s Development 

• Improving Public Education 

• Raising Academic Achievement and Persistence in College 

• Supporting Low-Wage Workers and Communities 

• Overcoming Barriers to Employment 

Working in almost every state, all of the nation’s largest cities, and Canada and the United 
Kingdom, MDRC conducts its projects in partnership with national, state, and local govern- 
ments, public school systems, community organizations, and numerous private philanthropies. 




